Pre-trained Transformers currently dominate most NLP tasks. They impose, however, limits on the maximum input length (512 sub-words in BERT), which are too restrictive in the legal domain. Even sparse-attention models, such as Longformer and BigBird, which increase the maximum input length to 4,096 sub-words, severely truncate texts in three of the six datasets of LexGLUE. Simpler linear classifiers with TF-IDF features can handle texts of any length, require far less resources to train and deploy, but are usually outperformed by pre-trained Transformers. We explore two directions to cope with long legal texts: (i) modifying a Longformer warm-started from LegalBERT to handle even longer texts (up to 8,192 sub-words), and (ii) modifying LegalBERT to use TF-IDF representations. The first approach is the best in terms of performance, surpassing a hierarchical version of LegalBERT, which was the previous state of the art in LexGLUE. The second approach leads to computationally more efficient models at the expense of lower performance, but the resulting models still outperform overall a linear SVM with TF-IDF features in long legal document classification.

我们考虑使用最新的MultieRlex数据集中考虑法律主题分类中的零射击跨语性转移。由于原始数据集包含并行文档，这对于零拍传输不现实是不现实的，因此我们开发了一个没有并行文档的数据集的新版本。我们使用它来表明，基于翻译的方法非常优于多绘制预训练的模型，这是多曲线的最佳先前的零弹性传输方法。我们还开发了一种双语的教师零摄像转移方法，该方法利用了目标语言的其他未标记文档，并且比直接在标记的目标语言文档上进行微调的模型更好。

感知毒性取决于会话上下文的用户帖子在目前的毒性检测数据集中是罕见的。因此，在现有数据集上培训的毒性探测器也将倾向于忽略上下文，在发生这种情况时使上下文敏感毒性更加困难。我们构建和公开发布10,000个帖子的数据集，其中有两种毒性标签：（i）注释者认为每个帖子作为上下文; （ii）注释者没有其他背景。基于此，我们介绍了一个新的任务，上下文敏感性估计，旨在识别如果也考虑上下文（前一篇文章），则识别感知毒性变化的帖子。然后，我们在此任务上评估机器学习系统，显示可以开发实际质量的分类器，我们表明，具有知识蒸馏的数据增强可以进一步提高性能。这些系统可用于增强具有更多上下文依赖的帖子的毒性检测数据集，或者建议当主持人应考虑父柱时，这通常可能是不必要的，否则可能会引入显着的额外成本。

Laws and their interpretations, legal arguments and agreements\ are typically expressed in writing, leading to the production of vast corpora of legal text. Their analysis, which is at the center of legal practice, becomes increasingly elaborate as these collections grow in size. Natural language understanding (NLU) technologies can be a valuable tool to support legal practitioners in these endeavors. Their usefulness, however, largely depends on whether current state-of-the-art models can generalize across various tasks in the legal domain. To answer this currently open question, we introduce the Legal General Language Understanding Evaluation (LexGLUE) benchmark, a collection of datasets for evaluating model performance across a diverse set of legal NLU tasks in a standardized way. We also provide an evaluation and analysis of several generic and legal-oriented models demonstrating that the latter consistently offer performance improvements across multiple tasks.

We study a novel and important communication pattern in large-scale model-parallel deep learning (DL), which we call cross-mesh resharding. This pattern emerges when the two paradigms of model parallelism - intra-operator and inter-operator parallelism - are combined to support large models on large clusters. In cross-mesh resharding, a sharded tensor needs to be sent from a source device mesh to a destination device mesh, on which the tensor may be distributed with the same or different layouts. We formalize this as a many-to-many multicast communication problem, and show that existing approaches either are sub-optimal or do not generalize to different network topologies or tensor layouts, which result from different model architectures and parallelism strategies. We then propose two contributions to address cross-mesh resharding: an efficient broadcast-based communication system, and an "overlapping-friendly" pipeline schedule. On microbenchmarks, our overall system outperforms existing ones by up to 10x across various tensor and mesh layouts. On end-to-end training of two large models, GPT-3 and U-Transformer, we improve throughput by 10% and 50%, respectively.

Cryo Focused Ion-Beam Scanning Electron Microscopy (cryo FIB-SEM) enables three-dimensional and nanoscale imaging of biological specimens via a slice and view mechanism. The FIB-SEM experiments are, however, limited by a slow (typically, several hours) acquisition process and the high electron doses imposed on the beam sensitive specimen can cause damage. In this work, we present a compressive sensing variant of cryo FIB-SEM capable of reducing the operational electron dose and increasing speed. We propose two Targeted Sampling (TS) strategies that leverage the reconstructed image of the previous sample layer as a prior for designing the next subsampling mask. Our image recovery is based on a blind Bayesian dictionary learning approach, i.e., Beta Process Factor Analysis (BPFA). This method is experimentally viable due to our ultra-fast GPU-based implementation of BPFA. Simulations on artificial compressive FIB-SEM measurements validate the success of proposed methods: the operational electron dose can be reduced by up to 20 times. These methods have large implications for the cryo FIB-SEM community, in which the imaging of beam sensitive biological materials without beam damage is crucial.

概念诱导是基于正式的逻辑推理在描述逻辑上的，已在本体工程中使用，以从基本数据（ABOX）图创建本体（Tbox）公理。在本文中，我们表明它也可以用来解释数据差异，例如在可解释的AI（XAI）的背景下，我们表明它实际上可以以对人类观察者有意义的方式进行。我们的方法利用了从Wikipedia类别层次结构策划的大型层次结构，作为背景知识。

故障自然是随机的，而大多数人造系统，尤其是计算机都可以确定地工作。这需要将概率理论与数学逻辑，自动机和切换电路理论联系起来。本文通过量子信息理论提供了这种连接，这是量子物理学遵守概率定律的直观方法。在本文中，我们提供了一种新的方法，用于计算使用基于门的量子计算机开关电路的诊断。该方法是基于将代表叠加错误的量子位放置的想法，并同时诊断出所有的量子，通常是指数级的。我们从经验上将用于诊断的量子算法与基于SAT和模型计数的方法进行比较。对于组合电路的基准，我们在估计故障的真实概率方面建立了不到百分之一的误差。

故事结束一代旨在为给定的故事背景产生合理的结局。该领域的大多数现有研究都集中在产生连贯或多元化的故事结尾，而他们忽略了不同的角色可能会导致给定故事的不同结局。在本文中，我们提出了一个面向角色的故事结束生成器（Coseg），以自定义故事中每个角色的结局。具体来说，我们首先提出一个角色建模模块，以从故事背景中提取的描述性经历中学习角色的个性。然后，受到化学反应中离子交换机制的启发，我们设计了一个新颖的矢量断裂/形成模块，以通过类似信息交换程序来学习每个字符和相应上下文之间的固有相互作用。最后，我们利用注意力机制学习有效的特定角色相互作用，并将每种相互作用馈送到解码器中，以生成角色 - 与角色的结尾。广泛的实验结果和案例研究表明，与最先进的方法相比，Coseg在生成的结局质量方面取得了重大改善，并且有效地自定义了不同字符的结局。

我们展示了一个端到端框架，以提高人造系统对不可预见的事件的弹性。该框架基于基于物理的数字双胞胎模型和三个负责实时故障诊断，预后和重新配置的模块。故障诊断模块使用基于模型的诊断算法来检测和分离断层，并在系统中产生干预措施，以消除不确定的诊断解决方案。我们通过使用基于物理学的数字双胞胎的平行化和替代模型来扩展故障诊断算法为所需的实时性能。预后模块跟踪故障进度，并训练在线退化模型，以计算系统组件的剩余使用寿命。此外，我们使用降解模型来评估断层进程对操作要求的影响。重新配置模块使用基于PDDL的计划，并带有语义附件来调整系统控件，从而最大程度地减少了对系统操作的故障影响。我们定义一个弹性度量，并以燃料系统模型的示例来说明该指标如何通过我们的框架改进。